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PREFACE 



This Technical Note is the first of a seven-part issue which 
describes a Cost-Effectiveness Model for the analysis of Title I 
ESEA Project Proposals. The seven parts are: 

TN -14 Part I - An Overview of the Cost-Effective- 
ness Model and Submodels for the 
Evaluation of Title I ESEA Proposals 

TN-15 Part II - The School Submodel 

TN -16 Part III - The Instructional Process Submodel 

TN-17 Part IV - The Community Submodel 

TN-18 Part V - The Cost Submodel 



TN-19 Part VI - The Effectiveness Submodel 
TN-20 Part VII - The Simulation 



The model was developed by Abt Associates, of Cambridge, 
Massachusetts, for the Division of Operations Analysis, National 
Center for Educational Statistics, under contract number OEC 1-6- , 
OOI 68 I-I 6 SI, 



DESIGN FOR AN 

EDUCATION SYSTEM COST -EFFECTIVENESS MODEL 

by 

Dr. Clark C. Abt 
Abt Associates Inc. 

Cambridge, Massachusetts 

This presents a design for an elementary and secondary 
education cost-effectiveness model, emphasizing evaluation of the U.S. 
jjlementary and Secondary Education Act*s Title I programs for the dis- 
advantaged. Substantively, the design reflects a concern for the exploit- 
ation of all available data on results accomplished by educational sys- 
tems, as well as what is known about learning and influence processes. 
Methodologically, the design attempts a quantitative description of edu- 
cation s ystems , that may be programmed as a computer simulation that 
will produce quantitative indications of the impact of a Title I project 
on the school, the students and the community. 

The model at this writing has been partly programmed for com- 
puter simulation, and empirical data are being collected for the vali- 
dation. 

The model was developed in 1966 under contract for the U.S. 
Office of Education*s Division of Operations Analysis, by an interdis- 
ciplinary team at Abt Associates Inc. , a private research firm located 
in Cambridge, Massachusetts. Some five man-years of professional 
effort were expended by fifteen professionals under the direction of the 
writ( 2 r. The writer gratefully acknowledges the inspiration and encour- 
agement given by Dr. Alexander Mood, Assistant Commissioner for 
Educational Statistics; the wise direction and warm support of Dr. 

David Stoller, Director of the Division .of Operations Analysis; Dr. 
Richard Powers, Chief of the Education Economics Analysis Branch; 
and Mr. Martin Spickler; all of the U.S. Office of Education. Signifi- 
cant parts of the model design are the work of my colleagues at Abt 
Associates Inc. , Stephen Bornstein, Louis Cutrona, Stephen Fitzsim- 
mons (deputy project director), Raymond Glazier, James Hodder, 
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Holly Kinley, Peter Miller (deputy project director), M. Keith Moor 
Martha Mulloy, Michael Pritchett, Robert Rea, Martha Rosen, and 
Richard Rosen. Professor Andre Daniere* and Mr. George Thomas 
{)£ Harvard University generously gave advice and information. 



THE OVERALL MODEL AND THE SUBMODELS ' 



The purpose of the overall education cost-effectiveness model ‘ 
is to evaluate the relative school, student, and community” effects 
and associated costs of alternative ESEA Title I programs for the dis- 
advantaged. 

Since such programs are directed toward increasing learning, 
the model focuses strongly on the changes in student achievement, 
the attitudes and environmental factors influencing achievement, and 
the social behaviors and community impacts of improved achievement 
in the target population. 

The model may be described as a micro-educational model, be- 
cause of its representation of some of the detail components of the 
education process. However, the model does not pretend to be a micro- 
. analysis of learning and influence processes, although these processes 
are represented by whatever objective correlatives are available in the 
form of qualitative numerical indices. 

The model also does not pretend to be an exhaustive representa- 
tion of what leads to changed student achievement, attitudes, carnlnf^ 
potential, and equality of educational opportunity. The attempt was to 
^^phasize those aspects of the education process that seem most rele- 
vant to achievement increases in students affected by Title I programs, 
and for which quantitative data is widely available. 

Some attitudinal variables believed decisive for the learning pro- 
cess are not yet quantitatively defined, and therein only qualitative, im- 
pressionistic data available on them. Rather than simply omit such 
troublesome but significant variables, and thus falsely imply insignifi- 
cance by omission, the qualitative variables are sometimes given nu- 
merical index ratings roughly corresponding to such qualitative distinc- 
tions as are offered by empirical but impressionistic data. In other 
cases, qualitative variables are built up numerically from components 
for which better data is available, or assigned index values by user, 
judgment. In all cases, the attempt has been made to achieve a useful 
balance among the demands of data input, model complexity and validity- 
of output. 
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The model’s emphasis is on what the education system produces . 
in terms of quantities and qualities, rather than how it does so. Howr 
. ever, a certain amount of detail on how it produces its effects was es- 
sential to simulate for forecasting what it will do. 

The model is not initially expected to be predictive, but only 
indicative of the relative cost-effectiveness of alternative Title I pro- 
grams, ‘Prediction requires regularity of process, and no two schools, 
student populations, or communities are alike. Even the calibration of 
the model with previous Title I before-and-after data will only improve 
its indication of the probable relative effectiveness of programs, because 
of the uniqueness of each case. Only to the : extent that Title I situations 
are similar and are accurately measured and modeled, can their impact 
be forecast. However, even such a limited cost-effectiveness forecast- 
ing and evaluation model as that described here offers a substantial aid 
to education planners and policy-makers. 

The overall model consists of five submodels: (See Figure I below) 

--School 

--Instructional Process 
--Community Interactions 
--Costs 

--Cost-Effectiveness 

The School Submodel represents the production process whereby 
the inputs of four partially educated students types (white and non-white, 
above and below $2, OOO family income) and education resources (teach- 
ers, equipment, facilities, community environment) are transformed 
into better educated individuals, graduates, and dropouts. Inputs include 
Title I programs, previous school achievement and student sociological 
data, and specific improved achievement in the target population. Out- 
•puts are the change in numbers of graduates, dropouts, achievement 
levels by grade and student type, and critical achievement areas. 

The Instructional Process Submodel represents the specific im- 
provements in student achievement and attitude resulting from a Title I 
program. It attempts to reproduce the effects of the influence process 
whereby behavior and attitude are modified by exposures to teacher. 
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INPUTS • • . • OUTPUTS 
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Figure 1: Cog>t- Effective ness Model 
Overview Showing Inputs, 5 Submod« 
els, and Outputs. 
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parent, and peer of varying duration and intensity. Inputs are the 
Title I program, previous and current changes in attitudes and axhieve- 
ments, and sociological data. Outputs are changes in achievement and 
attitude (’Index of Learning Difficulty’) by student type. 

The Community Interactions Submodel estimates the impact on 
seven community variables of the changes in education system output 
due to Title I programs. The inputs are the School Submodel outputs of 
changes in numbers of graduates, dropouts, and achievement levels, 

* the Instructional Process outputs of changed student achievement and at- 
titude; and community characteristics from the Data Base. Outputs <tre 
the community changes in terms of changes in lifetime earning potential 

and equality of educational opportunity. 

The Co st Submodel accounts for all direct and indirect costs re- 
quired to implement Title I programs. It allows the user to associate 
specific costs with specific components of the program and their effects. 
Inputs are the Title I program description and costs, and the real national 
average costs of typical Title I items. The submodel compares typical 
and proposed costs to allow the user to evaluate the efficiency of the pro- 
posed expenditures. Outputs are the total costs of the program, the 
added resources bought, and total program costs broken down by com- 
ponent parts. 

The Effectiveness Submodel is the submodel in which the analysis 
and the output of the results determined by the other submodels takes 
.place. The inputs to the Effectiveness Submodel are all of the variables 
which are in the Data Base at any time, the outputs of all of the above 
submodels (i. e. , student, school, and community effects), and their 
associated costs. Outputs are efficiency data, measures of education 
effectiveness, and descriptive school, student and community data. 
Specific efficiency measures are effects per cost, values per cost, 
effects per resource, and values per resource. 

The outputs of each Title I program would be evaluated by at least 
one model ’run, * and comparison of alternate programs can be made by 
comparing respective effects and costs. Both different programs for 
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the same target population, and the same or different programs for 
different target populations may thus be compared. The tables below 
are examples of typical computer printouts (output) of a model run for 
a single specific school improvement project, giving before-project 
and after-project achievement, attitude and economic data on students 
by population type. 
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PREFACE 



This Technical Note is the second part of a seven-part issue 
which describes a Cost-Effectiveness Model for the analysis of 
Title I ESEA Project Proposals. The seven parts are: 



An Overview of the Cost-Effective- 
ness Model and Submodels for the 
Evaluation of Title I ESEA Proposals 

The School Submodel 

The Instructional Process SuTmnodel 

The Community Submodel 

The Cost Submodel 

The .Effectiveness Submodel 

The Simulation 



The model was developed by Abt Associates, of Cambridge, 
Massachusetts, for the Division of Operations Analysis, National 
Center for Educational Statistics, Tinder contract number OEC 1-6- 
001681-1681. 
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THE SCHOOL SUBMODEL 



The School Submodel is similar to a production process with ^ . 
quality controls. The inputs of four student types (white ^nd non- 
whitCf above and below $2, 000 family income) and education re- 
sources (teachers, equipment, facilities, community environment) 
are transformed into better educated individuals, graduates and 
dropouts. Inputs include Title I . programs, previous school achieve- 
ment and student sociological data, and specific improved achieve- 
ment in the target population. Outputs are the change in numbers of 
graduates, dropouts, achievement levels by age or grade, and criti- 
cal achievement areas. 

. There are five subroutines: the school flow matrix, 

* course, of study selection and allocation, truancy and dropout, and 
graduation (see Eig, 1), The school flow matrix represents 
the flow of students through parallel achievement category tracks 
"sub-assembly production lines, " in a sequence of achievement levels 
within each track "quality control checkpoints." The achievement 
categories in various combinations form subjects such as math or Eng- 
lish. The achievement levels may represent grades or promotion thres 
holds (where schools are graded), but in any case are prerequisite 
gates for further achievement in the same and in related achievement 
categories. Any combination of achievement categories, achievement 
levels, and student types may be the target area of a Title I program 
(see Fig.' 2). 

At the eighth grade achievement level of the school flow matrix, 
the course of study selection routine is usually triggered. Students 
are allocated to college prep, clerical, general, or vocational high 
school courses of study, depending on their achievement levels and the 
entry requirements. * 

Truancy and absence rates are computed at all achievement or 
grade levels on the basis of external attitudinal and past achievement 
data. Dropouts are computed on the basis of past achievement, ab- 
sence, attitudes, and community factors. Graduates are computed on 
the basis of achievement at the final level or grade, and graduation crU 
teria. Finally, critical achievement areas, in terms of age /grade and 

subject and population, can be identified by iterated runs of the school 
flow matrix under various conditions. 
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THE SCHOOL FLOW NETWORK 
Introduction 



Forecasting long-range education effects as a result of 
Title I is accomplished in the school model by propagating changes 
in achievement resulting from the application of those Title I 
programs. The propagation starts afthe next highest grade and 
works forward by the successive application of a set of decision 
rules. The decision rules are based on predicted performance as 
a result of meeting curriculum prerequisites. The past perform- 
ance for each student type and course type is stored. The school 
flow. network, is overlayed upon the historic data. The achieve- 
ment change is then overlayed upon the network, propagated, and an 
• upgraded performance profile is generated. 



Description 

The school flow network consists of nodes at each grade/age 
achievement point. Nodes are connected by solid lines to indicate a pri- 
mary prerequisite exists in the previous grade for that achievement cate 
gory. Dotted lines indicate that the previous grade acts as a secondary 
prerequisite in that achievement category, 

. * grade, (g) 



(p) < 



Heading 
Language 
Soc. studies 
Math. 
Science 



The school flow network is used to infer the results of a 
Title I program on graduates and drop-outs. For example, if a 
Title I program is designed to improve the reading level achieve- 
ment of second graders, this improvement will result in an increased 
chance of improvement in most other fourth grade achievements 
(except possibly, *shop*). The network provides the prerequisites, 
and the historical achievement scores provide the basis for extra- 
polating ’downstream* effects. 
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Assumptions About Network Propagation 

The network is designed to operate in conjunction with a 
Title I program aimed at upgrading achievement. Several assump 
tions about operations are required: 



1) We assume tint in all cases: 



A A rt-ruh- js Aa , i 
^ Pi g-1 

is, a change in an/ achievement (A ) for 

p 



that 



any grade (g) requires the presence of a 
change in the same achievement at the 
previous grade. This is the horizontal 
prerequisite. 



2) We assume that, in all cases, ^ny lost 
requirement prevents a change in achieve- 
ment at the next level. This is the gating 
function effect. The result is that any node 
having more than one prerequisite must 
have a change present on all its entry lines to 
assure passing a change to the next higher 
level. (If only weak prerequisities are not 
present, we may reduce the probability of 
a change to the next level. ) 




3) The "ripple effect" cau die out after a few 
grades if insufficient nodes have been activated. 



. 4) A change will be generated whenever the level 

of the prerequisite is higher than or equal to the 

next level and the line is a diagonal. This assump- 
tion prevehTs~the change from "washing-out" furth- 
er, up the network due to loss of only one of the 
requisites. 
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In case A, the change was not propagated 

diagonally because the historic achievement 

indicates a lag in only the center subject* In 

case B, the change was propagated up by the 

earlier second grade achievement in the top 

subject, but was not propagated down because ' 

no network link was provided. 

» 

5) Only estimates of standard improvements 
are fprecast by the network. No'attempt has 
been made to analyze the historic data for en- 
hancing- effects which would allow faster achieve- 
ments due to superior facilities or teachers 
further up the network. In other words, the 
network is sensitive to the correction of achieve- 
ment gaps or deficiencies, but will not in its 
present simplified form reflect improvements 
from average to superior levels. 



Network Operation 

Each of the decision rules is applied at each of the network 
nodes beyond the nodes aifected by Title I. Aj:hievement perform- 
ance is predicted for each population segment carried in the model. 
Drop-outs, secondary school course of study requirements, and 
expected graduates are based on. the predicted performance. 



Display 

The “shadow effect*' of early achievement failure and its 
removal or reduction after Title I programs will be displayed 
as shown in Figure I. The numbers represent level of achievement 

in terms of grade. The x's are used for graphical purposes to re- 
present the shadow. * 
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FIGURE I 

LONG RANGE EDUCATIONAL EFFECTS 
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The shadow cast over the entire performance before headstart 
has been partially removed. Remedial Arithmetic Vi^ould further increase’ 
achievement but secondary school failure may still be possible due to low 
relative achievement and poor attitude toward school. 



Reading 

# 

Language 

f 

Soc. studies 
Math 



Science 









/ ' • ... 

THE USE OF ACHIEVEMENT TEST SCORES 

The achievement test scores to be used as data by the model (e. g, 
Standford Achievement Test, STEP, Iowa MP, etc.) will provide statistical 
distributions of scores by grade or age level. For example, the typically 
expected level of achievement of fourth graders in reading will be the mean 
level of achievement for the fourth grade. A student who scores, let us 
say, one standard deviation below the mean can be classified as a third 
grade reader; two standard deviations above might constitute a sixth grade 
reading ability. 

Each of the student types has a set of achievement scores (grade 
levels) for each grade determined from historic records. When the 'raw 
data are in terms other than grade level— for example, grades or percentiles— 
the scores are converted to grade level before being used in the model. 

The network pjropagation results are maintained for each student type 
independently and can be combined later for a presentation of mean achieve- 
ment and distribution by grade or age. Student types are defined as the bases 
of white (W) vs. non- white (N) and family income above $2, OOO vs, below $2, 000. 
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COURSE OF STUDY SELECTION SUBROUTINE 

Early in his high school years the student may choose a course of 
study designed to prepare him for college, for commerce, or for the voca- 
tions. Some students develop their curriculums within these courses of 
study while others select a general course of study. The ''Course of Study 
Selection Subroutine" indicates changes in Title I apportionment of students 
among the courses of study. 

After a Title I input, students are allocated among the courses of 
study by reconciling the demand for each course of study with the supply of 
slots in that course of study. The student choices are made from a list of 
those courses of study for which he has the required record of achievement. 

The student eligibility and preference for a course of study are based 
upon his characteristics. However, restricting the definition of student 
types to the average (mean) of the distributions of characteristics precludes 
simulating the process by which the various individuals within a student type 
select different courses of study. Therefore, it is necessary to distribute 
the characteristics which determine eligibility and preference along a dis- 
tribution the shape of which is a judgmental input. 

The minimum standards for eligibility are expressed in terms of re- 
quired XT’nimum levels of previous student achievement. The allowed student 
choices are those for which the student, generated from the distribution of 
characteristics for the student type, meets the minimum requirements. A 
list of allo.wed courses for that student is thereby created. 

The student makes his actual choices^from this list of allowed choice s> 
upon the basis of his attitude toward education. The course of study which is 
most compatible with the student attitude toward education is chosen. 

The choices of individual students are aggregated to determine the 
total demand for the available openings in each course of study. A check is 
made to determine if the demand for any course of study is greater than the 
supply, the excess demands are eliminated by allocating the appropriate 
number of students among the unfilled openings according to their second 
preferences for allowed choices. An output report is generated when there 
is unfilled demand. 



^fter supply and demand have been reconciled, or if the demand 
does not exceed the supply for any course, the changes in the apportion- 
ment of students among the courses of study is determined, by comparing 
the Title I apportionment with the historical apportionment. 
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THE TRUANCY SUBROUTINE 



The truancy subroutine computes an index of truancy for the 
grades 9 * 12. The truancy index is used to compute dropouts. 

Research Findings 

The following findings summarize the elements considered in our 
model of truancy. 

Self 

A study of truants (1) studied 338 truant students in San Francisco. 
Sixty percent of the students were boys. Their I.Q. *s averaged below 
normal. Average I.Q. for their schools were 100; their curve 95 (range 
43 « 153). Student had low opinions of self, felt isolated. 

School 

53% of students were in junior high school 
17% elementary 
30% high school 

70% had been left back at least once and were below average 
All achieved an average of-C grade or below. 

Home 

^ 45% of the students came from disrupted or broken. homes 

50% of families received public, assistance. 

Outcomes 

Truants were treated by case workers. Younger students in clcrnr n- 
tary levels improved their attendance rates after treatment. Older students 
showed no improvement. 

Summary 

Both this study and a British one (2) agree that the truant’s family 

situation is poor economically and interpersonally. The school situation is 

poor: the student performs below average grade-wise and has been left 

behind. He responds by being out of school approximately 3 times as often 

as the non-truant. 

• • 

(1) John h, Roberts, ’’Factors Associated with Truancy", Personnel and 
Guidance Journal , March, 1956, pp. 431-436. 

(2) Maurice J. Tyerman, "Research in Truancy", British Journal of Edu- 
cational Psychology, November, 1958, pp. 225-229. 



The. Model of Truancy 

« * 

The following figure depicts the factors considered in computing 
the index of truancy. The factors are weighted and combined into an index 
of likelihood of a change in truant behavior. The index is compared to a 
threshold and will determine the actual change passed along to the dropout 
subroutine. 

Equations for Computation 

f - 

The change in truancy for grades 9 - 12 caused by a Title I program 
is computed as: 




P cs 



cs 



where; 




change in truancy for student type (t) and grade (g) 




change in target group achievement by category (p) 
type (t), grade (g) and course of study (cs) 




change in target- group Self-Esteem 



= Scaling parameter 



a^, a^^ = weighting coefficients 
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DROPOUT SUBROUTINE 



The change in dropouts as a result of the application of a Title I 
program is computed for each of the grades between 9 and 12 for each of 
the student types. The change is basically the difference between the his- 
torical dropouts who attended the school affected by Title I, and the 
dropouts anticipated from the neighborhood school after Title I. 

Research Findings- 

The chief variables pointing to dropouts are: 

Home 

1. • Family of Dropout 

a. Dropout rate of parent 

One study revealed that 78. 5% of mothers dropped out. 

80. 3% of fathers dropped out. 

Majority of parents had completed 9th grade or less. 

(Percy V. Williams, '’Dropouts”, NEA Journal, 

Feb., 1963, 11-12.) 

« 

b. Dropout rate of siblings 

58% of dropouts come from families where all brothers 
and sisters dropped out, 

(Eli E, Cohen, "The Dropout Problerh, A Crowing Edu- 
cational Concern", National Association of Secondary 
School Principals Bulletin , April 1961. ) 

Occupation of Parent 

Majority of parents of dropouts were unskilled or unemployed. 

(Morris Williams, "What Are the Schools Doing about School 
Leavers", NASSP Bulletin, XXXVII, 1953, p. 54.) 

3. Divorce Rate in Family 

Many dropouts come from broken homes. 

School 

• Reading Competency 

• 

. Ruth Penty studied 1169 ninth graders entering 10th grade- 

593 in poorest quartile and 593 in top quartile as readers. 
(Poor readers ranged 4. 3 - 6.9 on reading test per grade 
level. ) 

49. 9% of the poor readers and 14, 5% of the good readers 
left school in the 10th grade. 



45. 5% of the poor readers graduated as compared to 
81.2% of the good readers. 

Dropouts peaked in 10th grade. 

(Ruth C. Penty, Reading Ability of High School I^ro pouts. 
New York, Teachers' College, Columbia University, 1956. ) 

5. Failure in Academic Subjects 
Most dropouts are failing. 

Most failures occur in 1st, 2nd and 9th grades. 

Dropouts Do Not Participate in extra curricular school 
activities. 

85% rated low in participation 
60% isolated 

(Livingston A. Hugh, "High School Graduates and Dropouts, 
A New Look at a Persistent Problem, " School Review 
LXVI (1958) 195-203) 

7. Attendance 

More truancy and more long periods of absences in 
dropout than graduate history, 

8. Transfer 

Many dropouts transferred from other schools.- 
More transfers than in graduate population. 

9. Dropout has few friends in school, associates with older 
youngsters and does not feel he belongs. 

Variables not important between dropouts- and non-dropouts; 

I. Q. » Penty study showed median I. Q. of dropouts average--not 
significant difference from graduating groups. 

(Bert I. Greene, Eastern Michigan University's Work Shop on the 
Dropout. Eastern Michigan University, 1963. ) ' 



Model of Dropouts 

The figure on thf? following page depicts the interaction of the basic 
factors used in the model. Community characteristics describing the 
economic advantages available in the community are compared with student 
attitudes and behavior to predict the change in dropouts. 
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. • • 

• Pyopout Decision Function 

We define the dropout decision function, © = 0(s) where s is any 
Student, and 

• . * 

0 > 0 implies that student s drops out, 

^ 0 implies that student s stays in. 

© « D(A) • (I^ Pc - Kj. Pj.) + 

* J* 

where A is the present age of student s; 

D(x) is the probability that a student who is going to drop out 
will drop out at age x; 

is the community dropout propensity; 

Pp is the student*s propensity to finish high school; 

and Kp,Kp, K^. are appropriately chosen constants, 

KjjVO, Kp>0. 
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GRADUATION COMPUTATION 

, ' — 

We wish to compute the expected change in the number of graduate# 
to be expected from increased academic achievement in the Title I target 
group* Basically* we need several historically derived sets of data. 

!• The numbers of last year’s graduates who were previously 
enrolled in the target population school by student type • 

2. The set of minimum acceptable achievement levels by category 
for graduation. 

. 3f The number of students directly affected by Title I by student 
type. 

4. The propagated achievements of the target group in the 
twelfth grade. 

The equation for the expected graduation change is: 



AgRAD^ = N^ . GR^ if 



Tf 

p,cs I 



DIM (A - G )1 

' p, 12, t,cs pM 



^ 0 



where: 

A GRAD^ = Change in graduates by student type (t) 

Nj = Number of students by type in the target population. 



GR^ = Last year’s graduates from the target neighborhood schools 

by student type. 

• • 

A lo 4 . ^ej “ Propagated Achievement by category (p), for the 
■* * twelfth grade ( 12), by student type (t), and by 

course of study (cs) 



Gp - Minimum acceptable achievement by category for graduation. 
DIM = Function which computes the positive difference or zero. 
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PREFACE 



This Technical Note is the third part of a seven-part issue 
which describes a Cost-Effectiveness Model for the analysis of 
Title I ESEA Project Proposals. The seven parts are; 
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An Overview of the Cost-Effective- 
ness Model and Submodels for the 
Evaluation of Title I ESEA Proposals 

The School Submodel 

The Instructional Process Submodel 

The Coinmunity Submodel 

The Cost Submodel 

The Effectiveness Submodel 

The Simulation 



The model was developed by Abt Associates, of Cambridge, 
Massachusetts, for the Division of Operations Analysis, National 
Center for Educational Statistics, under contract number OEC 1-6- 
001681-1681. 



THE INSTRUCTIONAL PROCESS SUBMODE L 

From the Cost-Effectiveness Model overv'iew it should be 
apparent that the principal thrust of a Title I . project is directed at 
the student himself. Notwithstanding the impact of Title I funds oh 
the community as a whole, the primary emphasis of Title I is the 
improvement of individual personalities and achievements. As a result 
a fundamental process in the overall Education Cost-Effectiveness 
Model is the determination of the differential scholastic effects of 
alternate Title I proposals. Thli* computation is carried out by the 
Instructional Process Submodel (tee Ciiart I), which accepts the 
changes in’ the school environment wrought by the proposed project, 
and converts these changes into expected educational outcomes. These 
outcomes are then sent on to the School Submodel, where they are aggre- 
gated and their effects projected over time, and similarly to the Community- 
Submodel where they affect the community propensities for social ^nd 

cultural change. Underlying the Instructional Process Submodel is 

< • 

the hypothesis that the culturally -disadvantaged child- initially fails in 
school because he is not sensitized to the scholastic environment. 

The student perceives his immediate world as the only relevant one, 
and that the language forms and cultural norms extant in the classroom 
are totally remote. He falls behind very early in his school career, 
receives no incentive to make up the gap, falls hack even further, and 
often drops out of the system altogether. Title I is devoted to the 
reversal of this trend, and sorne of its effects are estimated in the 
Instructional Process. 

Charts II and III illustrate the logic of the Instructional Process 
Model. Title I Project Description data is received by the School 
Submodel, and makes changes in both the curriculum and operating 
conditions. Then the School Submodel compares the pre- and post- 
Title I conditions, and the differences are used to evaluate the extent to 
which the target population will become mo/e sensitized to the academic 
environment. The sensitivity indicators are what may be called "Learning 
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CHART in 



catalysts, or accelerators of the learning process, These "catalysts" 
consist of a set of personality characteristics which, if stimulated at 
certain stages of the student's development, will greatly enhance his* 
educational productivity. (See Chart IV.) Expected change in student 
productivity is computed in Scholastic Achievements as measured by 
standardized tests. These test scores, in addition to the changes 
in learning catalysts, are the outputs of the Instructional Process 
Submodel, and the fundamental indicators of the Title I impact on tl\e 
student population (See Chart V). 

■ Clearly, the mathematical equations that transform changes in 
the environment to changes in student attitudes, personality, and 
achievements are difficult to formulate on the basis of current research. 
Consequently, these relationships will initially be calibrated with empirical 
data collected from school records, and, if possible, from Title I Case 
Histories, 

First, the curriculum node to be affected by Title I is defined 

(by subject, grade, student types, teacher /student ratio, and time 

allocation). Then, assuming the subject is not new to the school, the 

same curriculum node characteristics are defined for the previous year. 

If there are no differences between curriculum node descriptions for 

both the past and the present year, then the same amount of achievement 

change for each student type as occurred last year can be expected. 

On the other hand, if the student populations, or the curriculum contents, 

or the operating conditions, are different for the two nodes, then these 

differences can be correlated with the learning catalysts, and these in 

turn with scholastic achievements. The model will then compute the 

expected outcornes, and these can be checked with the actual achievements 

of the previous year's student population. The internal relationships 

among model variables can thus be calibrated by successive tunings 

until the predictions approximate the empirical results. When Title I 

Is introduced, these same relationships are assumed to hold, and 
• ^ 

environmental changes are sirnilarly CMivcrted into expected educational 

outcomes. 

♦ 

Stanford Achievement Test for Elementary and Secondary, Metropolitan 
Readiness for Kindergarten. 
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Schematic representations of the model logic are given in 
Charts VI and VII, In the first chart, student characteristics, 
operating conditions, and curriculum descriptions for pre- and post- 
Title I school environments are compared, related to the appropriate 
learning catalysts, and evaluated for their ultimate impact. In the 
second chart changes in learning catalysts are used to modify the 
historic or base-line achievement gains mentioned above. In both 
charts. Title I Projects have been subdivided into a "curriculum” 
category, for all subject matter related programs, and an "operating 
conditions" category, for all service related activities. Differences 
in student characteristics are determined by comparing the historic 
student populations with the Title I target group. 

A more detailed example of how a changed learning catalyst can 
affect scholastic achievement is presented in Chart VIII. If the catalyst 
called EMPATHY (the ability to tahe another person’s point of view) is 
stimulated between the ages of six and nine, the corresponding effects 
on three appropriate achievements (Mathematics, Social Studies, and 
Language) are computed from the empirically set relationships. The 
choice of appropriate ach'evements is made from both qualitative and 
quantitative research results, just as the choice of the appropriate 
catalysts affected by environmental changes is derived. These choices, 
and the relative weights associated with them, are subject to user 
judgment and modification. They also imply needed research areas 
and, since they are flexible, can be adjusted at any time with the dis- 
covery of more precise empirical results. 

The last chart (number IX) identifies the required input categories 
for the operation of the Instructional Process Submodel. There are four 
of these: Title I Project Descriptions, Title I Case Histories, Current 

School Data ^Historic School Data, and Research, Theory, and User 
Judgment. Each is given with some exemplary data and the source 
from which the data is derived. Special attention has been paid to data 
availability and an attempt has been made to exploit to the fullest the most 
accessible data, and to avoid the problems of sensitive data collection. 

To recapitulate, the problem for the ”lnstructional Process Submodel 
IS '|How much will each Title I Project affect the achievements and 
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CHART IX 




attitudes of the target population?'* The Instructional Process defines 
a set of learning catalysts which must be stimulated before any 
accelerated learning can take place. Title I projects change the 
curriculum, or the operating conditions, or the student population, 

.or combinations of these, and these changes activate the learning 
catalysts. If the catalysts are increased above a given' threshold, 
then they will in turn affect scholastic achievements. These changes 
in scholastic achievements represent the impact of the Title I Project 
on the student population. 
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PREFACE 



This Technical Note is the fourth part of a seven-part issue 
which describes a Cost-Effectiveness Model for the analysis of 
Title I ESEA Project Proposals. The seven parts are: 



TN-U Part I - 



TN-15 


Part II - 


TN-16 


Part Ill- 


TN-17 


Part IV - 


TN-18 


Part V - 


TN-19 


Part VI - 


•111-20 


Part VII- 



An Overview of the Cost-Effective- 
ness Model and Submodels for the 
Evaluation of Title I ESEA Proposals 

The School Submodel 

The Instructional Process Submodel 

The Community Submodel 

The Cost Submodel 

The Effectiveness Submodel 

The Simulation 



The model was developed by Abt Associates, of Cambridge, 
Massachusetts, for the Division of Operations Analysis, National 
Center for Educational Statistics, under contract number OEC 1-6 
001681-1681. 
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THE COMMUNITY SUBMODEL 



The Community Submodel as it now stands is a set of independently 
operating subroutines "which convert I>r?.ta Base information. Instructional 

Process Submodel attitudinal arid School Submodel achievement 
data into indicators of selected community factors in which a change is 
expected or desired ( See Chart C-1 )• These outputs are selected on the 
basis of presumed interest to Title I evaluators. Present and past values 
for these outputs for a particular community will be already known; the 
Title I evaluator's concern is the predicted future values which incorporate 
both Title I inputs, tracing their community effects, and e>rt:rapolated 
community trends contributing to a change in these indicators relatively 
independent of Title I. 

For long-range planning it would also be desirable to cycle these 
outputs back through the community to see how they affect the School and 
the Instructional Process (presumably a sort of 'snowball* process). A 
complete full-scale working model of the community would perform all three 
desired f-unctions. However, data gaps and the absence of systematic 
research findings on the majority of the many components of a complete 
Community Model leave too much to speculation, ^ The current Community 
Submodel represents the alternative of higher reliability on a somewhat 
more limited scale. The two criteria for selection were relative reliability, 
i.e. , areas in which extant research has a"t least pointed the direction of 
relationships, and interest for immediate Title I evaluation needs. 

Each of the seven submodel outputs, with the exception of the 

comiposite community involvement indicator, has a separate subroutine 

logic, although they share many of the same Instructional Process inputs. 

The logics produce target population community effects, with and without 

Title I changes in Instructional Process variables used. 

• 

♦This finding is based on a full-scale theoretical modeling effort carried 
cut earlier in the history of this contract and reported in the Appendix. The 
results of this feasibility study are nevertheless generally favorable, given 
the possibility of a larger scale effort with an associated research program. 
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COlvlMUNITY SUBMODEL _ ^ 

C«2 

DYNAMIC OVERVIEW 
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The future value predictio-^ for Community Submodel outputs, and 
thus .for community changes (Z^’s) resultant of Title I inputs, are derived 
from the process common to all of the simulation. (See Chart C-2 for the 
dynamics). The main implication of the first Base Line Rim for the 
Community Subroutine logics is the establishment of threshold values; this 
is the 'tuning* of the submodel. Once these threshold values are established, 
the Model is run v/ith the projected Title Ipr gram inputs. Resultant 
changes in Instructional Process achievements and attitudes are passed on 
to the Community subroutines where they are combined in various ways and 
tested for threshold value. The result is a set of projected propensities 
for community effects on the part of the target population. By subtracting 
the Base Line Run outputs from the 'future' outputs we get the changes 
resultant from the particular Title I program being evaluated, These 
effects are Independent of other changes due to extensions of current 
community trends or exogenous inputs other than Title I. Hence the out- 
put.*^ are in terms of tendencies rather than predictions of specific behavior. 



EARNING POTENTIAL SUBROUTINE 
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TRANSLATION OF OCCUPATION TO INCOME C-3 a 




EARNING 

PREDICTION 

DATA 



EARNING PCTENTXAL SUBROUTINE 



The Earning Potential Subroutine is a simple but reliable 
procedure for quantitatively assessing the impact of any proposed Title I 
program upon the ability of the target population to earn a living. Its 
operation is straightforward: predict on the basis of course of study 
(Academic, Commercial, General, Vocational), achievement level in 
school, and graduation /dropout statistics, that occupational class 
(Business/Professional, White Collar /Clerical, Skilled, Semi-Skilled, 
Unskilled) in which each of the distinguishable student groups within 
the model will earn a living (Fig. C-3). From this prediction, and 
from correlation study data relating occui>ational class with expected 
lifetime earnings, we can compute a rough figure for total predicted 
earnings of the model population. Comparing total predicted earnings 
figures from the base line run and the run with Title I programs, we can 
get an index of change in Community Earning Potential, i. e. , large 
increase, increase, no change, decrease, large decreases (Fig. C-3a). 

In point of fact, the procedure described above is one part of 
a subroutine created for the current project which models the economic 
life of a community in greater detail. While this subroutine is at aTevel 
of detail too high for the purposes of the current overall model, a 
description is included of it as Appendix A to indicate the direction which 

f 

might profitably be taken should a decison to expand the scope of the 
Cofnmuni^ Submodel be taken. 
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SQUALITY OF EDUCATIONAI. OPPORTUNITY SUBROUTINE C-4 
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EQUALITY OF EDUCATIONAL OPPORTUNITY 

I 

The primary aim of Title I is to equalize educational opportunities 
throughout the country. Since ’’opportunities’* themselves are difficult, if 
not impossible to measure directly, this subroutine utilizes Coleman’s^ 
logic in deriving an indicator of the equality of educational opportunity. 

His method was to correlate achievement scores with social origins. 
Where equality of educational opportunity was high, there would be no 
correlation with social origin. 

This subroutine computes a mean achievement score for each of 
the four population groups, based on data from the School Model, It then 
prints out a distribution of means and computes the range. If the range is 
small, a high degree' of equality of educational opportunity is indicated, 
and the opposite is shown by a large range. Any Title I program which 
succeeds in raising the achievement level of any of the population groupfe 
whose mean is low will reduce the inequality of educational opportunity. 




^James Coleman, ’’Equal Schools or Equal Students”, The Public Interest, 
Vol, 4, Summer, 1966, 
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EFFICACY SUBROUTINE C-5 




I. P. 

NEED FOR 

ACHIEVEMENT 



SOCIAL STUDIES 
ACHIEVEMENT 



LANGUAGE 

ACHIEVEMENT 



I. P. 

LEVEL OF 
CONFIDENCE 
IN A LOGICAL 
WORLD 



I. P. 

PERCEPTION OF 
SOCIO-ECONOMIC 
OPPORTUNITY 





NO LOW 

^ EFFICACYl 





HIGH 

EFFICACY 
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EFFICACY SUBROUTINE 



The fvmction of the efficacy subroutine is to produce an indicator 
of a population group's power over its environment and hence its 
ability to bring about a desired result. It implies" both the willingness and 
ability to effect change, and a belief that ordered change and improvement 
are possible. In evaluating the effectiveness of a Title I program, it is 
worthwhile to examine changes in a community's likelihood for undertaking 
such activities as self-help projects. This routine translates attitude and 
achievement indices of the target population, (outputs of the instructional 
process and school models) into a series of decision thresholds. Each 
threshold acts as a screen to filter off those population groups who do 
not meet minimum success criteria at each level of decision, 

i 

• For example, a Title I sponsored work- study program in which 

students could apply newly acquired skills to remunerative work might 
increase their need achievement level. If this were greater than the 
established threshold, one would then examine their organizational ability. 
For the purposes of the model, social studies achievement and language 
skills, determined by standardij-f. i i cores and aggregated by population 

type are used to ’yield an index of organizational ability. Such a work- 
study program could enhance social studies achievement but might not affect 
communication skills. If the population group did not meet the minimum 
requirement for organizational ability their efficacy index would be low. 

If their organizational ability were adequate, two attitudinal variables 
would be considered: the population group's level of confidence in a 

logical world and their perception of their socio-economic - opportunity. 

Unless there exists a high level of . confidence in a rational, ordered world 
where, given the proper channels, one can effect change, a population group 
would not attempt a community action project. Both of these attitudes 
miglit be enhanced by a work- study program by acquainting the students 
with a real-world example of cause and effect in the work situation and 
be increasing their socio-economic level. The population groups who 
successfully pass through the four decision thresholds and have the pre- 
requisite organizational ability will receive a high effica.cy rating. 
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SOCIAL PARTICIPATION PROPENSITY SUBROUTINE 



For present model purposes the function of this subroutine is the 

production of a social participation propensity output which is defined 

as formal organizational participation (voluntary associations like civic 

groups, fraternal orders, as well as churches, political parties, labor 

and commerce associations, etc. ) The importance of this factor to 

community involvement, especially as a sort of marker of the traditional 

American cultural pattern, has been recognized since it was first noted 

in DeToquevilie's Democracy in America . We therefore expect it to 

contribute to the expected effect of Title I on the community. 

Within the subroutine certain attitudinal indices for the 

target population are input from the Instructional Process Submodel; they 

are combined and tested for threshold value to produce social participation 

propensity. A positive social participation propensity value is ’multiplied’ 

by the availability of formal organizations, a data base input. 

Need for affiliation is a necessary but not sufficient input. This 

need is compared with the calibrated threshold need value. If it is greater 

than threshold, we have satisfied the need condition and move to the 
♦ * 

capability condition. Capability here is social skills sufficient to enable 

one to join formal organizations. Social skills equal interpersonal skills, 

. An index of social skills is computed from Empathy and Self-Esteem 

attitudinal variables (inputs from IP). These two factors are both deemed 

« 

necessary in some degree for successful social interaction; a high degree 
of both self-esteem and empathy is considered^piuch more favorable than 
a high on one and low on the other . « 

Therefore the scale values are multiplied to insure that an 
extraordinarily high value on one does not compensate in the index for 
‘an extraordinarily low value on the other, as might occur were this an 
additive functioii. 

Example: 

Self-Esteem .. | 1„.., \ i — I ' 

1 .10 

I I ■ .1 

1 . . 10 



Empathy 



Sample values: 


SE = 6. 


SE = 4 




EM = 2 


EM = 4 




'Fair' 


‘Good* 


If additive; 


SE + EM 


= Social Skills Index 




6 + 2 = 8 
4 + 4 = 8 


Can’t distinguish 



If multiplicative: 

SE X EM = Social Skills Index 
6x2 = 12 ‘Fair’ 



4x4 = 16 ‘Good* 

Social skills in^ex is then tested for threshold value. If the 
threshold is passed, the result is ‘medium* social participation propensity. 
Propensity, however, also includes some measure of availability, in this 
case possible formal organizations per population. An increase in number 
of formal organizations extant (available) per population therefore' results 
in high change in social participation propensity. 

A Title I program involving field trips for cultural and educational 
development might be expected to raise Empathy directly, and indirectly 
^increase Self-Esteem, thus contributing greatly to the Social Skills Index 
which' multiplies the two. It is therefore a good example of how Title I 
can produce considerable effects in social participation propensity. 
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PROPENSITY FOR COMMUNITY INVOLVEMENT SUBROUTINE C-7 
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PROPENSITY FOR COMMUNITY INVOLVEMENT 



One of the important factors in maintaining or achieving 
community cohesiveness is the level of involvement or vestment various 
population groups may have in the community. The propensity for 
community involvement is a combination of a group’s propensity for 
cuitural and social participation, as described above, and their desire 
and ability to act, or their efficacy rating. The indices derived from 
these three subroutines are weighted and summed to provide an index of 
community involvement. 
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CUXiTURAX. PAKTXCIPATION SUBROUTINE 

Cultaral participation propensity is defined in this model as a 

whilh^r” acquire the values, beliefs and norms 

wh h therr community has dra.n from the nationally shared culture. 

Thts subroutme serves two purposes in the model. First, it provides a 

oug m ication of the extent to which an exogenous Title I program may ' 

of ZU ul^- ■ — -^er 

and ff T with social participation propensity 

and efftcacy, this subroutine yields an indicator of community involvL::: ^ 

, The process for determining cultural participation propensity is 

aaerres of decision thresholds .rough v Mch each target popltion pisses 

pe mg an mdicator of the level of pariicipation. The theory underlying 

th *'*'"**' ^ curiosity greater than an established 

0“ ^«^ 4 uate 
natio 1 , “ P°P"l^«on group will have the tendency to share in the 

na c ture. Pending achievement level is included because it is the 

best avauable mdicator of the kinds of mass media a group may utilise. To 
incMHed 7 ”tr^d reading achievement are ' 

ness:;ti:iras? 

propensity is also increased . 

For example, assuming an adequate curiosity threshold, let us 

IZT: ^ Mgh 

ge with incomes of less than $2, 000. If it does not succeed in 

.ncreasmg the reading skill of this population to sixth grade level, tCir ' 

rldfo^IlTtete'' Pa^^^cipation will be low and probably include only ‘ 

.this program^ , 

and ■ V ® achievement to this level, newspapers 

magazines become possible sources of information. If reading achieve- 

nt IS at the eighth grade level, then the^propensity for cultural participakon , 
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will be high, provided adequate facilities are available. A judgruental 
input IS required to determine ivhether current facilities are saturated 
K the Title X program increases cultural facUiiies. a negative respll!: 
at this decision threshold ivould yield a high propensity for cultural 
participation. It is possible for the subroutine to show that additional 
facilities may themselves attract new participants and hence enhance 
the cultural participation propensity rating for their population group 
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SOCIAL CONFLICT SUBROUTINE 
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SOCIAL CONf:uICT SUBROUTINE 

This subroutine is intended to generate a propensity for destructive 
social conflict in. the community. While there is without doubt a certain 
level of social conflict implicit in increased community involvement our 
concern in this subroutine is v,'ith disruptive, ‘unhealthy’ conflict levels. 

The subroutine output is a rough indicator of propensity for high level 
social conflict; it is not sufficiently detailed to give scale values. 

A community history judgment of incidence of destructive social 
••^ conflict is input to the subroutine from the Data Bank. A recent precedent 
of destructive social conflict is one of the three contributors to social 
conflict propensity, although like the others it is not absolutely necessary. 
This input is not affected by Title I. 

I 

Socio-economic grievances are a second factor. It should be 
noted that actual socio-economic inequalities are not as relevant as 
perceptions of socio-economic opportunity. Means by population group 
of perception of socio-economic opportunity (from the Instructional 
Process Submodel) are compared by taking the range of the means, 
an indicator of the spread of the disparity. The range • threshold is set 
by model c'alibration. If, at a later date, research demonstrates that 
this part of the subroutine has greater predictive value, we may 
be able to connect size of range of the means with specific levels 
(scope and intensity), rather than employ the present simple threshold 
test (YES, NO). 

In addition we need an attitudinal indicator of aggressive action 
orientation. A rough index is computed from the Identification With 
Authority (not orientation to authority) and confidence in logical world. 

The two scales are multiplied in accordance with the hypothesis that a high 

’f'This variable is a more general version of "Negative Attitude toward 
Science" found by Adorno, et. al. , ( The Authoritarian 'Personality, Harper, 
1950, p. 464) to be highly correlated with high Ethnic Prejudice Scores 
and to some extent with aggressive behavior. The advantage in using 
"confidence in worldly logic" is that it is less tied to factual scientific 
knowledge. 
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score on both is necessary for an aggression index. This index is computed 

for each population group so that high values will not be washed out by 

aggregation. A high aggression index value for any group is considered 

* • 
sufficient to contribute to community social conflict. 

Keeping in mind the roughness of these three indices, we consider 
the presence of a ”YES” on any one factor an indication of low propensity, two 
»'YES*s" medium propensity, three "YES's” high propensity. 

School-job coordinators input to the School Model is a good 
example of how Title I may affect disparity of perception of socio-economic 
opportunity. Good job coordination guidance could be expected to raise 
low means for certain groups of perception of socio-economic opportunity. 

Similarly a school-job coordinator or other Title I program may 
affect "confidence in a logical world" and by increasing the denominator 
of the Aggression Index computation lower the index value. If either or 
both of these changes lowers the index value to less than the threshold value, • 
we can expect a reduction of propensity for social conflict. 
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CHIME dUB-ROUTINE 

^ * 

Results from empirical research attempting to establish 
universal or area causes (e. g. personality based, physiologically based) 
oi criminal behavior have been uniformly disappointing. Researchers and 
theoreticians have been unable to isolate clear cut causes of crime. It is 
felt that this is due to cultural and individual differences which interact in 
differing ways. Attempting to'cut across such differences may well involve 
the summation of variables which negatively correlate with one -another, 

-/ thus cancelling out each other when aggregated. 

A second approach which does not attempt to establish causative rc- 
, * lationships seeks to determineconditions under which criminal behavior is 

most likely to emerge. Here correlations rather than causes are the'basis • 
of examination. The Crime Sub-Routine makes use of information about cer- 
tain variables which appear to be significant corollaries to crime. Based 

• • • 

upon the literature the variables are divided into two groupings: Community 
and Student characteristics. Then variables in each of the sub-groups are 
weighted as contributing either 1 or 2 points. Data is insufficient to justify 
using any stronger mathematical treatment than summation. It is assumed 
that the greater the number of conditions favorable to crime for a defined 
population, the greater the probability that criminal behavior will occur for 
a segment of that population. Thus, when conditions cumulate to 15 or more 
points on a 20-point scale, the outcome is probabilistically assumed to be 
criminal behavior. Where such is the case, exogenous OE programs likely 
to deal with the corollaries to crime are recommended; following development 
of such programs, the student population characteristics are updated and re- 
turned to the Data Base and crime propensities recalculated. 

Use of the scale requires the user to determine the characteristics of 
his community as described. If for example his school system is in a neighbor- 
hood with high-density but well kept homes, he would score 2 points for Housing 
Density and 0 for Deteriorated Housing. The maximum score for all community 
characteristics is 10; likewise for student characteristics. Where the sum of 
all propensity indicators exceeds 15, the Crime Decision Function outputs an 
increased propensity for criminal behavior. 
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The following studies provide the rationale for the variables included 
in the Community and Student Characteristics boxes: High Housing Density, 
High Deteriorated Housing, High Tr inM. at Population, High Ethnic Hetero- 
geniety and High Social Upheaval arc all reported by Freedman et al, 1956> 
as being correlated with criminal and delinquent behavior in the city. It is 
in areas so characterized that a disproportionate amount of crime is perpe- 
trated, and where the greatest number of convictions are generated. Aside 
from empirical studies of correlations, police records support these, variables 
as corresponding to the individual criminal behavior. These groups, it may be 
speculated, are also most vulnerable to the pressures of organized criminal 
activity. The existence of a double standard of justice (as for example with 
Negro populations) is considered by many psychologists and legal specialists 
as both reinforcing criminal behavior, and as a basis for disregard of civil 
rights by police, leading to further reinforcement. 

Turning to the more individually oriented variables contained in the ’ 
student characteristics box, the following empirical support exists. Raab 
and Selznick, 1959, found that low identification with society as seen in broken 
homes and rharriages, lack of integration with neighborhood, etr., was signi- 
ficantly correlated to delinquency rates. Sykes, 1956, reports that frequently 
the problem of delinquency rate is a function not of deviate behavior from a 
total society, but rather of conformity to a primary peer group which promotes 
behaviors inconsistent with society; i.e., a cultural- subculture discrepancy. 
Crime rates are also very high among adolescents, particularly among males, 
as contrasted with other age groups (and females)— See Cressey, 1961. Work 
by Cohen and Short, 1961, indicates that among minority groups and particu- 
larly Negroes crime is disproportionately high. 
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PREFACE 



This Technical Note is the fifth part of a seven-part issue 
which describes a Cost-Effectiveness Model for the analysis of 
Title I ESEA Project Proposals. The seven parts are: 
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THE COST SUB MODEL 

In order to assign meaningful cost-effectiveness weightings to 
a proposed Title I programi the Cost Sub-Model has been developed. 

This model is designed to compute the total cost of a Title I program 
including both direct and indirect expenditures required for the program, 
Actual average costs of direct requirements, such as reading machines, 
teachers^ salaries, salaries of health personnel, audio-visual equipment, 

. • etc. will be stored in the data base of the model for every Title I 
program which the model will 'he able to handle, ^ 

In addition, costs of supporting requirements, such as an 
empty classroom and additional administrative personnel often- 
times available within the school system itself, though not necessarily 
taken into consideration by the user, will be stored with each program 
package. Hence, when a Title I program is introduced into the Cost 
Model, not only its direct requirements, but also its supporting require- 
ments will be examined. As illustrated, by Figure II, the model will com- 
pare current resources available in the school with their utilization rate to 
arrive at an evaluation of resources available to fulfill supporting require- 
_ ments for the Title I program. Thus, when the cost computation is 
executed, total cost of the project is obtained, not just the obvious cost 
of the major portion of the program in question. 

The methodology to be used by the Cost Sub-Model in determining 
this total cost figure is outlined in Figure III, Both direct and indirect 
purchases required for the program package are listed, including both the 
initial expenditure required for the first year of operation and the dis- 
counted present value of the future costs where applicable. These costs 
are printed out for the benefit of the user according to the format in 
Figure HI, Total cost of the package is also sent to the cost-effectiveness 
sub-model as an input to be used in calculating relative cost-effectiveness 
measures. 
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Compute Needed Resources and Cost 







Once the total cost of the program has been determined, this 
cost estimate will be compared with the cost estimates prepared by the 
user in those cases where he has already estimated the cost of the project, 
^id discrepancies between the two estimates will be listed for the user. 

In addition, the total cost of the program will be compared with the 
total supply of funds available for the project and if the cost estimate 
exceeds the expenditure limit, the computer will stop and indicate- 
this fact to the user. (Refer to Figure IV) Both of these diagnostic 
routines serve as a check to the user when the cost of the program is 

significantly greater than the amount of money he had originally planned 
to expend. 

The cost sub-model is integrated with both the instructional 
process and exposure sub models. The program package being evaluated 
is fed into the instructional process sub model after its characteristics 
are translated in a series of descriptors able to be processed by the 
instructional process model. The exposure sub-model receives the 
new. resources which the cost model indicates .are required to execute the 
Title I program in question, and updates the data base accordingly. 

The exposure model also supplies the cost model with the number of 
students included in a particular target group selected by the user as 
the recipients of the Title I program, thereby allowing the cost model 

to determine the quantity (and thereby costs) of resources required in 
any given Title I program. 
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■ THE EF3TSCTIVBNESS SUBMODEL, 

♦ • ^ 

The overall model will simulate particular school districts and 

their output in terms of student changes and community impacts. This 
simulation is done, in particular, by the School submodel, the Instruc- 
tional Process submodel and the Community submodel. The Cost sub- 
model determines the costs of the various Title I inputs and the on-going 
school expenses. It is the job of the Effectiveness submodel to re- 
ceive the outputs of these four submodels and the updated data base and 

♦ 

provide output for- the user. * * . ■ 

What kinds of outputs are there? 

Chart 1 shows two kinds of outputs which are generated by the sim- 
ulation. If v/Q operate the model without any Title I programs (the base 
♦ ^ 

• run), the output will be in terms of pre-Title I levels; this type of student 

had an achievement level in math. of such-and-such, for example. These 
pre-Title I levels are important for testing the model and tuning the judg- 
ment parameters in it - this is the first kind of output. 

After making the base run, we will run the simulation with a pro- 
■ posed Title I program present. The output of this run will be another set 
of levels of student change and community impact, “*but these are not very 
useful by themselves. The information we seek describes the impact of 
the possible Title I program on the school. The only way'to determine 
this is to compare the base run with the Title I run. 

• X 

Thus, the second kind of output derived from the simulation is a 
set of changes (or deltas) for each of thie variables of interest,. Let us 
examine the two kinds cf outputs in detail, and see what they consist of. 

Baseline output * - . 

Chart 2 shows the baseline- output for a target population consisting 
of the fourth.grr.de in a particular school. Described are: . 

The target population " • ’ 

• • . -A set of aggregate results 

• ' Results broken down ih' detail • 
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BASELINE RESULTS 



TARGET POPULATION: 4th GRADS 

‘ AGGRSdATE RESULTS - GRADE 4 

i\CHIEVEMENT LEVEL (I.O.) 

READING LEVEL 

SOCIAL ADJUSTMENT 

. ■’ MATH SKILL 

LANGUAGE SKILL 

PERSONALITY DEVELOPMENT 

$ 

EMOTIONAL CONTROL 



95 

3rd GRADE 
POOR 
■ POOR 
POOR 
FAIR 
POOR 



DISTRIBUTED RESULTS » GRADE 4 

■ ■ TYPE NUMBER 




STUDENT POPULATION 



CHART 2 - BASELINE OUTPUT 
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Thio baseline output shevvi; the target population at a particular 

point in ti:re . li is a cnapshot o£ the fourth graders; its source is the 

instructional process model. Another type of baseline output is that 

« 

.provided by the school model. Chart 3 shows the curriculum flov/ new;ork 
for the school in question and lets us see what the expected future is for 
the children in question without Title I inputs. 

Chart 4 shows the delta results derived from the simulation and the 
outputs which are drawn from them. We are interested in the effects which 
occur as a result of the introduction of the Title I input. ( Effects and 
Resources). However, this sort of output may not be very useful for 
comparison of alternative Title I programs, since different resources 
are likely to be used by different programs. It is therefore necessary.. to 
convert resources into some common unit; the most obvious unit is that 
of dollars. V/e can then calculate the effect per dollar change; a useful 
number. For the same reason. that we converted resources into common 
units, it is necessary to convert effects into common units; that is, we 
want to compare alternative programs with different effects and resources 
along some common dimension. 

The unit for the comparison of effects is value. The value for 
• • 

changes in the various effects is of course a judgment input - this input • 
may be obtained from the user by means of the Churchman-Ackoff approx- 
imate measure of value procedure or other -means. Given changes in effects 
weighted 'by their associated values and the resource changes described in 
•terrno of dollars, we can derive the necessary index of value/dollar derived 
* from the Title I program. 

In the case -v/here only one resource (e. g. , textbooks, teachers, 
schoolrooms, etc.) is being introduced by a Title I program, the amount of 
a value resulting from an increased- unit of that resource may be a desir- 

able output. 



V/hat do these outputs look like? 

Chart 5 shov;s the information needed to determine the incremental 
value per dollar .which results when a Title I-program is introduced into a 
school. The total cost of the program, the magnitude of the effects derived 
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•The shadow cast over the entire performance before headstart 
has. been partially removed. Remedial Arithmetic would further increase 
achievement but secondary school failure may still be possible due to low 
relative achievement and poor attitude toward school. 
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CHART 5 - VALUE/$ TABLE 



and the relative velucG of in the eftecto variables are necessary. 

The value-weighted changes arc surarned and this total is divided by the 
total cost to give value /dollars. Chart 5A is a graphical representation 
of this information. 

Chart 6 shows how effects and total cost ma.y be related. 

In Chart 7, we see the combination of Effects and Resources in the 
case where more than one resource has been used. This output is essen- 
tially a list of effects and resources vnth little demonstration of trade- 
offs. Chart 8, on the other hand, gives a possible output which v/e might 
SCO when only one resource has been usee in a Title I program, m each 
case, the simple calculation of value and resource is appended to the 
output. 
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CHAR 



CHANGEuS GREATER THAN 1% LN VARIABLES 
(EFFECTS AND RESOURCES) BETV/EEN BASE 
RUN AND RUN WITH TITLE I PROGRAM 



EFFECTS: 



INCREASES 



• MATH ACHIEVEMENT 
SOCIAL ADJUSTMENT 

DECREASES ’ 

LA.NGUAGE SKILL 



PERCENT 

•i- 10% 

+ 5%- 



2 % 




INCREASES 

. TEACHERS 
AUDIO-VISUAL DEVICES 



+ 10% 
+ 20% 



(TOTAL CHANGE +54 VALUE UNITS) 



CHART 7 - EFFECTS AND SEVERAL RESOURCE INPUTS 
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Kow does the simulation operate, to produce these results? 

Chart 9 shcr/ s the simulation and its operation. For further detail, 

% 

on this rather technical side of the cost-effectiveness model design. See 
the memo written by Hodder and Miller which covers the subject in more 
detail. 



A problem which may be of more interest to the user is: how does 
the simulation look to me? Chart 10 describes the interaction of the user 
and the computesr from the data- gathering stage until the final output is re- 
ceived from the sirnulation. This chart also is described in the memo 

A 

referenced above. 

If the user desires more detailed information on the simulation pro- 
cess in general or the three parts of the computer program shown in Chart 
9, -this detail will cds 0 be' found in the simulation memo. There are 
flowcharts which show, in more detail, the operation of each of the three 
parts of the computer operation of the overall evaluation tool. 
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PREFACE 



This Technical Note is t-he seventh part of a seven-part issue 
which describes a Cost-Effectiveness Model for the analysis of 
Title I ESEA Project Proposals. The seven parts are; 



TN-14 



TN-15 

TN-16 

TN-17 

TN-18 

TN-19 



TN-20 



Part I - 

Part II - 
Part Ill- 
Part IV - 
Part V - 
Part VI - 
Part VII- 



An Overview of the Cost-Effective- 
ness Model and Submodels for the 
Evaluation of Title I ESEA Proposals 

The School Submodel 

The Instructional Process Submodel- 

The Community Submodel 

The Cost Submodel 

The Effectiveness Submodel 

The Simulation 



The model was developed by Abt Associates, of Cambridge, 
Ifessachusetts, for the Division of Operations Analysis, National 
Center for Educational Statistics, under contract number OEC 1-6- 
001681-1681. 
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THE SIMULATION 
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1. Introdaction 

The simulation v^ill be used by the Office of Education to evaluate . 
programs in particular communities. It will not be used explicitly to allo- 
cate all Title I money in an optimal way; the model will not tell us to spend 
.$300, 000 in district A, $20, 000 in district B, and so on for all of Title I. 
Rather it will tell the user that for community A, program 1 yields better 
results for less money than program 2. The simulation will be an evalua- 
tion and planning tool, not a research instrument. 

The model will compare the effects and costs of programs or com- 
binations of programs with a set of basic effects and costs which derive from 
the school districts* operation without changes. In computer terms, we will 
compare alternative program runs with the base run. 

Although research use is not the goal of our model development effort, 
it will be necessary for the user to have the opportunity to investigate in 
depth the results of program's and the v/orking of the model. This option for 
in-depth investigation will enable tlic user to develop his confidence in and 
facility with the model to whatever level of detail he desires. The user may 
have a particular area of interest in which he wishes to observe changes in 
variables which are not summarized in the print-out for the ordinary user. 

He should be able to investigate these changes. 

The three areas of concern for the simulation are: 

1) Data base preparation, 

* V 

2) Model operation, and 

3) Cost-effectiveness analysis and presentation of results. 

These three areas are linked by three executive systems: 

1) The Data Base Management System, 

2) The Executive Control System for model operation, 'and 

3) - The Analysis Control System for cost-effectiveness compu- 
tation and output display. 

Each of the three executive systems handles- the computer operations of data 
input, sequencing of activities, 'and generation of output. 
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Figure I shows these three systems separated by dashed. horizontal 
lines. The Title I prpposal and community data are connected to punched 
cards, examined for error, combined with the' judgment parameters set by 
the designers, and assembled on the data base input tape, After model- 
operation, the collected data is analyzed and user reports are produced. 
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2. User Interactions 



O 



The simulation has been designed to maximize the quality of the data 
used. Editing and analysis routines have been provided to check for errors 
of omission^ inconsistencies, and reasonableness of both the Title I and 
community data. 

Outputs are prepared automatically' when exceptions occur, pinpointing 
source data errors. Model and Cost-Effective results are presented as 
overviews, but the simulation allows further in-depth analysis upon request 
without rerunning the model. 

Community data is saved on the data base input tape. Title I results 
^re saved on the simulation output tape. These two tapes separate the three 
main simulation areas. Operation of the data base management system results 
in the data base input tape, simulation operation results in the simulation output 
tape, and analysis results in user displays. 

Figure II depicts the interaction points and shows the sequence of • 
events (vertically), the source of the operation (field, data processing, or user), 
the actual operation performed (data preparation, system operation, or data 
analysis), and the major stop points (horizontal dotted lines). An example of a 
typical sequence might be as follqws: A Title I proposal has been received and 
is to be evaluated. The proposal data is key punched and checked and an error 
report is produced. The errors are corrected and the data base tape is prepared. 
This tape is saved in case other Title I proposals are made for this school dis- 
trict. The. model is then operated and a Simulation Output Tape is generated. 

This tape is also saved to allow in-depth analysis of results at a later time. 
Analysis of generated data takes place using the Output Tape and reports are 
produced for the Title I target population. After user analyses of these reports, 
a request may be made to produce rori.ain in-depth data. In this event, a control 
card containing the data request will produce further specified reports. 
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USER INTERACTION DIAGRA*.^ 
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3. The Data Base Management System 



The management system performs three basic functions: 

1) Cost Analysis of the Title I proposals; 

2) Editing of the keypunched community data; 

3) Preparation of the data base input tape* 

Figure III illustrates operation of the management system. The Edit and Cost 
Analysis programs are operated on the punched cards to produce exception 

•reports. This prevents erroneous data being used by the model, and wasted 
computer runs. 

After these decks have been corrected, they are entered on the data 
base input tape and summary profiles are reported to the user. The profiles 
summarize the data by target student population, by target curriculum, and 
by other Title I impacts. 

3. 1 Cost Analysis 

The cost analysis compares pre-stored item costs with proposed 
expenditures and computes the total program cost for later use in the Analysis 
Executive Control System. 

3.2 Edit 

The edit functiorx picks up invalid keypunched characters, omissions, 
incorrect values, and inconsistent data. An exception report listing each in- 
correct card is produced to isolate problems. 

3. 3 Preparation of Data Base 

The data base contains two basic kinds of data: empirical data des- 
cribing the community; and generated data based on statistical distributions, 
for example, student characteristics. A profile generator c reates student, 
teacher, and curriculum profiles to be used by the model. The data base 
■generator combines these profiles, with the empirical data, and the Title I 
profile (the effects represented by the Title I request), and with othjer model 
coefficients, constants, and costs to create the data base tape. The tape con- 
sists of several records. Record I contains the basic cost data to be used by 
the Analysis Executive. Record II contains the base-line data base. Records 
III to N each contain the specific Title I profiles associated with the proposal. 
A control variable (NRUN) indicates the number of Title I profiles to the 
Executive Control System. 
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4. A Method for Operating; the Simulation 

There is a need to make each simulation run rapidly and efficiently 
in order to analyze many communities and many proposals at minimum cost. 

One way to achieve this is to operate the simulation only over the subset of 
the total school district directly affected by the Title I proposal. The re- 
quirement for data with which to operate the simulation will be directly 
affected by this operation. Only micro-data specific to the proposal target 
group need be collected from the school system.. For example, a proposed 
remedial reading program for the elementary school will not require the 
collection of all Junior and Senior High School statistics. The school super- 
intendent will be required to collect only the data needed to support his proposal. 

4. 1. Implications for Simulation Operation 8t Display 

Exposing only the target population to the Title I programs implies 
added constraints on the simulation control system. Parameters must be 
proyided to adjust operation of the simulation chosen target group. The 
simulation display can be tailored directly to the Title I target group. 

Simulation control must be able to expose as small a population 
group as a single classroom, and as large a population group as the entire 
school district. Decision rules for aggregating displays for the latter case 
are needed, to prevent exposing the user to a large amount of data. 

4, 2 Trade-offs and Costs 

Consider two alternative methods of operating the simulation: 

1) Focussed simulation runs exposing only the target group, 

2) Fixed runs exposing the entire school district. 

Focussed Simulation Runs increase the cost of construction, and decrease the 
cost of data collection to provide increased flexibility of use and short run 
time. Fixed Simulation Runs decrease the cost of construction, maintain a 
high fixed cost of data collection, but decrease the flexibility and speed of • 

f 

use. Alternative one affords direct user benefits and is therefore recom- 
mended. 
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5. Executive Control System 



The executive control system performs three principal functions: 

Run control. Model operation, and Simulation result recording. Figure IV 
represents the Control System operations. Input is from the data base input 
tape and output is recorded on the Simulation Output tape. The cost-data 
recorded in Record I of the data base input tape is written as Record I on 
the output recording tape for later use by the analysis control system. Suc- 
cessive files on the output tape represent sequential run results. The fiijst 
run is defined as the baseline . There is a run for each Title I proposal. 

The output tape provides a history of simulated results for each community 
both with and without Title I programs. 

5. 1 Run Sequencing 

The function of run sequencing is to reset the model data base with 
the proper settings in order to execute each of the comparative simulation 
runs. The base-line run is followed by runs incorporating each of the Title I 
proposals. The data base is restored from the. data base input tape. Specific 
Title I profiles are added and the model is operated. 

5. 2 Model Operation k Recording 

The function of this aspect of executive control is the sequencing of 
the model subroutines and data base recording. The order of model routines 
is: Target Group Exposure, Instructional Process, School Network, Com- 
munity Effects at the end of each sequence the data base is recorded on tape. 
Separate parameters are used to identify the particular target groups who will 
be exposed to the Title I programs. 
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6. Analysis Executive Control System 

The analysis system provides routines to evaluate the simulation 
results in terms of educational effects, costs, value of the effects, and 
resources. Comparison of the effects with Title I are compared with the 
base line to provide an indication of the value. of each program proposed, 

• The value of the program is -divided by cost to yield an indication of 
cost-effectiveness. Figure V illustrates the sequence, input is derived 
from the Simulation Output tape, and all output is in the form of user 
reports. . 
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,S» Estimated Computer Operating Time for Education Cost-Effectiveness 
' Model 



1. General 



The current design is presently estimated to have about a five 
minute operation cycle for each Title I proposal per community, on the 
RCA 3301 computer. This excludes data base makeup and analysis of data. 
The contributions to the five minute cycle are: 

Initialization 
Instructional Process 
School Network 



2. 0 min. 
1 . 0 min 
. 5 min 



Recording 



1. 5 min 



5. 0 min 



2. The School Network 



The timing is dependent on two factors: The number of network 
nodes or cells, and the number of ope rations /node or cell. The number of 
nodes is ^ 3 courses of study x 2 income groups x 2 races x 6 achievement 
categories x n grades affected 

or 72n nodes. 

We can estimate 100 instructions /node @ 5 ^sec/instructor. 



Hence: . 036n seconds. 



\ 





n 


Time 


Headstart 


12 


1/2 second 


Fourth Grade 

4 


8 . 


1/3 second 


Secondary School 


4 


1 /6 second 



The actual network operation time is not the critical factor. Rather 
the critical time factors are: calibrating the instructional process, ini- 
tializing the data base from tape, and recording data on tape. * ' . 
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